Below you can find some useful UNIX commands for exploring your next-gen data. You can find a more in depth UNIX tutorial here.

Clicking on the command will take you to its man page.

Getting Started | Man Pages | Navigating the File System | Mounting | Working With Text Files

---

Getting Started

clear or Ctrl-l: clear screen

up and down to navigate through previously used commands

Ctrl-r let's you search the command history of your shell

## Ctrl-r opens this query tool
## start typing part of the command you issued a while back and it will show
## up in the query tool -- I think the history gets deleted after logging out
(reverse-i-search)`':

use Tab to complete names of files and directories

exit or Ctrl-d: log out of shell

poweroff: one of several ways to shut down a computer from the shell prompt

Man Pages

man: displays the manual for the command you are interested in
To search a man page type / search term. To scroll through your search hits use n. To exit type q.

man man
## displays the man-page of man ;-)
man ssh
## displays man-page of the ssh client

Navigating the File System

Here is a nice introduction to the Unix file system. Reading this should clarify greatly how the file system on a typical Unix system looks like (Mac users may have to make some adjustments).

cd: move between directories
cd .. will take you back one directory in your path. You can use multiple .. arguments. For instance, if you are in /home/cartwrightlab/desktop and would like to get back to /home use cd ../..

pwd
/home/cartwrightlab/desktop
cd ../..
pwd
/home

cp: copy files

cp dir1/foo.txt dir2/
## copies foo.txt from dir1 to dir2
cp -r dir1/ dir2/
## copies dir1 and its contents to dir2

ls: list the contents in a directory

ls -l
## list files with detailed info
ls -a
## show all files including hidden files
ls -lh
## show a list of files including their sizes in human readable form
ls -lha
## well...

mkdir: make a new directory

mkdir foo
## makes directory foo

mv: move file or rename file

mv foo.txt newdirectory/foo.txt
## moves foo.txt into newdirectory
mv foo.txt cat.txt
## renames foo.txt to cat.txt
 

rename: rename multiple files according to a regular expression indicating a rule for renaming

rename 's/\.txt$/\.fasta/' *
## changes the file extension for all files (*) in a directory from .txt to .fasta

rm: remove files or directories

rm foo.txt
## delete foo.txt
rm -r foo
## delete the directory foo and its contents

find: search for files -- here's a great tutorial for harnessing the full power of find

find /home/user -name pattern
## searches for files in /home/user that match the pattern

df: report file system disk space usage

df -h
## reports the size, used space, and available
## space of every device available on the system
##
## useful if you want to know how much space you
## have left to work with -- Can I really extract this tarball?!

du: estimate file space usage / directory size

du -h /home/user
## estimates the size of the user's home directory
## and the contained directories
##
## can be used to estimate the size of any directory

pwd: print the path of the directory you are in

Note: All commands can be issued on the files you would like to work with from the directory they are in. They can also be issued from another directory. In this case use the path for your files relative to your working directory. Ex: Your file (file_1.txt) is in the directory home/cartwrightlab/Documents/TruSeq/ you are working in home/cartwrightlab/Documents and want to rename file_1 without leaving your current directory. The command would look like this:

mv TruSeq/file_1.txt TruSeq/newfile_1.txt

Notice that the entire path is not necessary since TruSeq is a subdirectory of Documents.

Mounting

mount: mount a device

A device can be a hard-drive. These are generally found in the directory /dev. The exact name differs from system to system. Let's say /dev/sdb1 is the USB drive you just plugged into your computer. You would mount it to the directory /mnt by doing the following:

mount /dev/sdb1 /mnt

umount: unmount file system

To unmount the drive from above example do this. The process may take a while to finish, as there may be data that has to be written to the device first!

umount /mnt

Working With Text Files

For additional information on working with text files see Editing text files.

less: print file contents to standard out; you can search patterns in less in the same way as the man pages

less foo.txt

head: print the first 10 lines of file to standard out

head foo.txt

to print the first 5 lines:

head -n 5 foo.txt

tail: print the last 10 lines of file to standard out

tail foo.txt

to print the last 5 lines:

tail -n 5 foo.txt

nano: text editor that is very easy to use
If all you would like to do is look at a file use less instead of nano. Nano is not ideal for editing very large files, see the editing text files post.

nano foo.txt

cat: concatenate files and print to standard out

cat foo.txt
## dump foo.txt to standard out (the screen)
cat foo1.txt foo2.txt foo3.txt
## dump foo1.txt through foo3.txt to standard out
cat *.txt
## dump all text files in current directory to standard out

grep: pattern search within file -- returns the lines in a file that contain the search pattern

## Simple pattern matching and full-fledged support for regular expressions.
## There are tons of how-to's on the web.
grep pattern foo.txt
## basic usage to look for pattern in file foo.txt
cat foo.txt | grep pattern
## same as above using cat and a pipe (see I/O Redirection)
cat *.txt | grep pattern
## search for a pattern in all text files in the present directory

uniq: report or omit repeated lines

cat foo.txt | uniq > foo-filtered.txt
## only retain uniq lines in foo.txt and save these in
## foo-filtered.txt
##
## think about filtering out duplicate sequences etc.

vi: powerful text editor with steep learning curve. Why would you learn the basics of vi? Because it's installed on pretty much any Unix(-like) computer you will encounter. Cheat sheet for reference.

wc: newline, word, and byte counts for each file

wc -l foo.txt
## counts the number of lines in foo.txt
cat foo.txt | wc -l
## same as above using a pipe (see I/O Redirection)